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ABSTRACT 


Three  experiments  were  run  using  the  SCHOLAR  CAI  system  to 
teach  geography  to  high-school  students.  The  experiments  compared 
a  method  of  teaching  derived  from  analysis  of  human  tutors  (Tutorial 
Mode)  vs.  a  method  derived  from  programmed  instruction  (Block-Test 
Mode).  In  the  three  experiments,  Block-Test  Mode  was  systematically 
converged  toward  Tutorial  Mode  in  order  to  pinpoint  what  aspects 
of  teaching  strategy  affected  student's  learning.  Tutorial  Mode 
was  significantly  more  effective  in  the  first  two  experiments,  and 
nonsignif icantly  in  the  third.  The  results  indicated  that  the 
major  factor  affecting  student's  learning  was  the  strategy  that 
tutors  use  of  reviewing  the  material  in  greater  uepth  on  a  second 
pass.  Allowing  the  students  to  ask  questions,  and  the  tutorial 
strategy  for  relating  new  material  to  the  student's  previous  know¬ 
ledge  contributed  only  a  small  amount  to  the  differences  found  in 
the  first  two  experiments. 
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INTRODUCTION 

When  Carbonell  (1970)  developed  the  SCHOLAR  CAI  system,  he 
created  a  new  type  of  generative  CAI  system,  capable  of  carrying 
on  a  tutorial  dialogue  with  students.  In  SCHOLAR  knowledge  about 
geography  is  stored  in  a  semantic  network  (Quillian,  1968)  of 
facts  and  concepts,  structured  like  human  memory  (Collins  and 
Quillian,  1972a).  Dialogue  with  the  student  is  handled  by  a  variety 
of  procedures  that  utilize  the  knowledge  stored  in  the  semantic 
network.  Different  procedures  can  be  used  to  ask  the  student 
questions,  evaluate  his  answers,  correct  his  errors,  answer  his 
questions,  and  present  him  with  new  information.  Because  the 
information  in  SCHOLAR  is  distinct  from  the  procedures  used  for 
teaching  that  information,  it  is  possible  to  vary  the  teaching 
strategy  while  holding  the  information  constant.  This  makes  it 
possible  to  compare  different  teaching  methods  in  a  systematic 
manner. 

Such  an  evaluation  can  be  mads  by  attempting  to  teach  students 
the  same  domain  of  information  with  different  versions  of  SCHOLAR, 
and  measuring  their  learning  by  comparing  pre-test  and  post-test 
scores  on  the  material  covered.  In  the  three  experiments  reported 
here,  we  compared  two  different  methods  of  teaching  in  SCHOLAR,  one 
derived  from  analysis  of  tutorial  dialogues  (called  Tutorial  Mode) 
and  one  derived  from  programmed  learning  (called  Block-Test  Mode) . 
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In  order  to  implement  a  Tutorial  Mode  in  SCHOLAR,  we  studied 
actual  human  tutorinq  of  South  American  geography  to  see  how 
teachers  adapt  their  teaching  to  the  individual  student  (Ccllins, 
Warnock,  and  Passafiume,  1975).  Several  important  findings  from 
this  earlier  study  related  to  the  ways  that  tutors  select  topics, 
interveave  questioning  and  presentation,  and  review  previous 
material.  Toqether  with  the  ability  to  answer  student  questions, 
these  aspects  are  oa sac  to  the  way  Tutorial  Mode  operated  in  the 
three  experiments. 

The  topic  selection  strateqy  used  by  the  best  tutors  produced 
a  structure  of  topics  and  subtopics  like  an  outline  for  a  course. 

For  example,  the  tutor  might  start  off  with  a  question  like  "Dc 
you  know  any  geographical  features  of  South  America?"  If  the 
student  nenticned  the  Andes,  for  example,  then  the  tutor  would 
discuss  various  aspects  of  the  Andes  for  a  while,  perhaps  including 
as  a  subtopic  the  highest  mountain  in  the  Andes,  named  Aconcagua. 
After  exhausting  the  important  information  under  the  Andes,  the 
tutor  would  usually  ask  about  other  geographical  features,  like 
the  Amazon  or  Cape  Horn.  Each  of  these  would  be  discussed  for  a 
while  until  the  major  geographical  features  were  covered,  at  which 
point  the  tutor  would  pick  a  new  topic  such  as  regions  or  countries. 
Thus  the  topics  and  subtopics  form  a  nested  outline  structure, 
with  the  tutor  probina  a  little  way  into  each  subtopic,  and  then 
popping  up  to  the  previous  level  when  the  important  information 
is  exhausted  at  the  lower  level. 
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the  essence  of  how  the  tutor  relates  his  teaching  to  tne  inaiviou«. 
student.  The  dialogues  showed  that  the  tutors'  questions  occur  at 
the  top-level  and  beginning  topics  in  the  outline.  This  is  because 
the  tutor  starts  out  asking  questions  to  find  out  what  the  student 
already  knows,  and  then  presents  new  material  that  is  related  to 
the  student's  previous  knowledge.  The  object  seems  to  be  to  tie  a 
much  information  as  the  student  can  assimilate  into  the  structure 
of  his  previous  knowledge  (Collins  and  Quillian,  1972b;  Norman, 


1973) 


The  other  important  aspect  of  tutoring  that  we  have  implemented 
in  SCHOLAR  is  reviewing.  In  the  dialogues  we  analyzed,  the  better 
tutors  went  over  the  material  on  a  second  pass,  asking  about  things 
the  student  didn't  know  the  first  time  through,  and  adding  more 
detail  to  the  structure  of  information  built  up  on  the  first  pass. 
The  tutorial  method  as  a  whole  reflected  a  strategy  Norman  (1973) 
refers  to  as  web  teaching,  where  the  teacher  first  tries  to  esta¬ 
blish  a  framework  of  basic  knowledge  and  then  fills  in  more  and 
more  detail  on  subsequent  passes,  much  like  a  spider  spinning  a 


In  contrast  to  the  tutorial  method  of  teaching,  the  strategy 
used  in  programmed  instruction  involves  presenting  small  amounts 
of  information,  and  then  asking  questions  about  the  information 
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just  presented.  We  built  a  version  of  this  strategy  into  Block- 
Test  mode,  which  first  presents  a  block  of  information  followed 
by  questions  about  that  block.  Block-Test  Mode  follows  the  same 
topic  selection  strategy  as  Tutorial  Mode,  but  unlike  Tutorial 
Mode  the  student  can  not  ask  questions. 

The  programmed  instruction  strateqy  in  various  forms  is  used 
widely  in  CAI.  But  there  is  a  fundamental  problem  to  this  method 
of  instruction:  in  order  for  a  student  to  answer  most  of  the 
questions  he  is  asked,  he  ends  up  half  parrotinn  back  something  he 
read  a  little  earlier  in  slightly  different  form.  There  is  at 
least  some  evidence  in  t.ie  psychological  literature  that  this  leads 
to  little  long-term  retention.  In  particular  Craik  (1971)  and 
McCabe  and  Madigan  (1971)  found  that  in  list  learning,  information 
recalled  from  the  end  of  the  list  is  unlikely  to  be  remembered 
later,  because  it  is  recalled  out  of  short  term  memory  without  the 
effort  necessary  to  retain  it  lunger.  A  similar  effect  in  paired- 
associate  learning  underlies  the  advantage  of  the  anticipation 
method  (where  presentation  of  the  first  member  of  the  pair  precedes 
presentation  of  the  pair)  over  the  prompting  method  (where  presenta¬ 
tion  of  the  pair  precedes  presentation  of  the  first  member) .  This 
is  because  the  prompting  method  involves  repeating  back  what  one 
has  just  read.  These  results  suggest  that  the  way  Tutorial  Mode 
combines  questioning  and  presentation  may  provide  an  advantage  over 
Block-Test  Mode.  In  the  context  of  CAI  such  a  tutorial  strategy 
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is  only  possible  in  a  generative  system  like  SCHOLAR. 

In  order  to  explore  what  aspects  of  the  two  teaching  strategies 
affect  students'  learning,  we  conducted  a  aeries  of  three  experi¬ 
ments.  Block-Test  Mode  was  converged  toward  Tutorial  Mode  during 
the  course  of  the  experiments  in  order  to  pinpoint  what  differences 
between  the  modes  had  the  major  effects  on  students'  learning. 
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METHOD 

This  section  describes  the  way  the  two  modes  operated 
ir.  the  experiments  and  the  testing  procedures  used.  Three 
experiments  were  run  altogether ,  each  with  eight  high-school 
students.  Each  student  lescned  about  two  South  American 
countries  in  Tutorial  Mode,  and  two  countries  in  Block-Test 
Mode.  Thus#  students  served  as  their  own  controls.  Students' 
learning  was  measured  by  the  difference  between  pre-  and  post¬ 
test  scores  on  a  pencil  and  paper  test. 

The  best  way  to  understand  how  the  two  modes  operated  is  to 
compare  protocols  from  the  two  modes  with  the  part  of  the  semantic 
information  network  on  which  the  protocols  are  based.  Figure  1 
shows  part  of  the  entry  under  Chile  that  was  used  in  the  protocols 
shown  in  Figs.  2  and  3.  Any  topic  stored  under  Chile  has  an  entry 
elsewhere  in  the  semantic  network  and  can  become  a  topic  in  its 
own  right,  as  happened  with  the  Central  Valley  in  Figure  2.  The 
entry  for  the  Central  Valley  shown  in  Figure  1  illustrates  how 
each  of  the  things  referred  to  under  Chile  are  expanded  elsewhere 
in  the  network. 

What  should  be  stressed  about  the  data  base  is  the  outline 
structure  of  information,  and  the  importance  tags  or  I-tags  (10  to 
16)  associated  with  each  fact.  The  structure  of  information  in 
the  data  base  parallels  the  nested  structure  of  topics  and  sub- 
topics  selected  by  the  better  tutors.  The  topic  selection  strategy 
described  was  therefore  applied  directly  to  the  data  base  structure. 
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CHILE 

SUPERCONCEPT  (10)  COUNTRY 
LOCATION  (10) 

IN  (10) 

SOUTH/AMERICA  (10)  SOUTHWESTERN 
BORDERING  COUNTRIES  (II) 

NORTHERN  (12)  PERU 
NORTHEASTERN  (12)  BOLIVIA 
EASTERN  (II)  ARGENTINA 

BOUNDARY  (12)  ANDES 

COAST  (II) 

WESTERN  (II)  PACIFIC 

ON  (10)  HUMBOLDT /CURRENT 
SOUTHERN  (13)  ANTARCTIC 
CAPITAL  (10)  SANTIAGO 

REGIONS  (II)  $L  CENTRAL/VALLEY  ATACAMA/DESERT  SOUTHERN /ANDES 
PEOPLE  (12) 

LANGUAGE  (10)  SPANISH 
RELIGION  (12) 

PRINCIPAL  (10)  CATHOLICISM 
POPULATION  (13) 

APPROX  (10)  9,000,000 


CITIES  (13) 

PRINCIPAL  (10)  $L  SANTIAGO  VALPARAISO  CONCEPCION 

ANTOFAGASTA  ARICA  VINA/DEL/MAR  PUNT  A/ARENAS  IQUIQUE 


centralA'alley 

SUPERCONCEPT  (10)  VALLEY  REGION 
LOCATION  (10) 

IN  (10) 

CHILE  (10)  CENTRAL 
ON  (10) 

COAST  (10) 

OF  (10)  PACIFIC 

CLIMATE  (12) 

TEMPERATURE  (10)  TEMPERATE  MEDITERRANEAN 
PRODUCTS  (12) 

AGRICULTURAL  (10)  $L  WHEAT  GRAPES  WINE 


I 

I 

I 


Fig.  1.  Partial  Entries  for  Chile  and  Central  \  .illey  in 
SCHOLAR'S  Data  Base. 
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The  I-tags  reflect  the  varying  degrees  of  importance  of  different 
facts  i\  the  data  base,  the  most  important  having  the  tag  10. 

These  guide  the  topic  selection  strategy  in  picking  the  most  impor¬ 
tant  information  at  each  choice  point,  and  in  deciding  when  to 
pop  up  to  more  important  topics.  The  $L  that  occurs  in  the  data 
base  indicates  that  the  following  elements  form  a  list. 

Tutorial  Mode 


In  the  experiments.  Tutorial  Mode  selected  topics  ^.n  oi.der 
of  importance .  It  started  out  asking  the  student  question*  about 
each  topic  (e.g.,  the  location  of  Chile  or  the  people  of  Chile). 

As  long  as  the  student  answered  correctly,  it  went  deeper  into  the 
topic  until  time  was  used  up  for  that  topic  or  there  was  no  impor¬ 
tant  information  left  (as  measured  by  I-taga) .  But  whenever  the 
student  made  a  mistake,  SCHOLAR  presented  a  few  related  facts  and 
popped  up  to  the  next  most  important  topic.  In  the  first  two 
experiments  students  were  allowed  to  ask  questions  whenever  they 
liked.  In  all  the  experiments,  time  was  allocated  between  a  first 
pass  over  the  material  and  a  review  pass,  where  questions  missed 
earlier  were  asked  again.  If  there  was  time  left  over  after  the 
second  pass,  SCHOLAR  would  start  on  a  third  pass .  The  material  was 
covered  in  more  depth  on  each  subsequent  pass. 

Figure  2  shows  an  excerpt  from  an  actual  protocol  on  Chile 
between  Tutorial  Mode  and  a  high-sr.hool  student  in  Experiment  2. 
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Dialogue 

WHAT  IS  CHILE? 

•COUNTRY  IN  SOUTH  AMERICA 
VERY  GOOD. 


Annotated  Comments 

This  is  the  most  mportant 
property  of  Chil'-.  (A 
tutor  would  suppress  it  on 
the  assumption  that  the 
student  knows  the  answer.) 
In  evaluating  the  answer# 
it  looks  for  "country"# 
ignores  the  rest. 


CORRECT  OR  INCORRECT?  Location  is  t  e  next  most 

CHILE  IS  IN  NORTHERN  SOUTH  AMERICA.  important  attribute  under 

Chile . 


•INCORRECT* 

YOU  ARE  CORRECT. 

CHILE  IS  IN  SOUTHWESTERN  SOUTH  AMERICA. 


The  student  answers 
correctly#  so  SCHOLAR  will 
go  deeper  into  Location. 


THE  EASTERN  BORDERING  COUNTRY  OF  CHILE 
IS 

•I  DON'T  KNOW* 


This  is  the  most  important 
Bordering-Country  of  Chile. 


THE  ANSWER  IS:  ARGENTINA. 

THE  BOUNDARY  OF  THE  EASTERN  BORDERING 
COUNTRIES  OF  CHILE  IS  THE  ANDES. 

THE  NORTHERN  BORDERING  COUNTRY  OF  CHILE 

IS  PERU. 


Since  the  student  has  missed 
the  question,  SCHOLAR 
presents  souk*  related 
information  ^bout  Bordering- 
Coun tries . 


THE  vORTH EASTERN  BORDERING  COUNTRY  IS 

BOLIVIA. 


WHAT  IS  THE  CAPITAL  OF  CHILE? 

•WHAT  ARE  SOME  CITIES  IN  CHILE* 

THE  PRINCIPAL  CITIES  OF  CHILE  ARE 
SANTIAGO,  VALPARAISO#  CONCEPCION# 
ANTOFAGASTA#  ARICA#  VINA  DEL  MAR# 
ARENAS#  AND  IQUIQUE. 


SCHOLAR  would  have  gone  on 
to  ask  about  Coasts  under 
Location#  but  it  has  run 
out  of  time  for  Location# 
so  it  goes  to  the  next 
attribute.  Capital. 

The  student  gets  some  help 
PUNTA  though  without  directly 

asking  SCHOLAR'S  question. 


Fig.  2a.  An  Annotated  Transcript  of  the  Beginning  of  a  Dialogue^ 

Between  Tutorial  Mode  of  SCHOLAR  and  a  High— School  Stwao'.t 
in  Exp. 2.  {The  Student's  Inputs  are  Between  Asterisks.) 


BBN  Report  No.  2885 


Bolt  Bcronek  and  Newman  Inc. 


NOW  YOU  CAN  EITHER  ANSWER  MY  QUESTION 
OR  ASK  A  HEW  ONE. 

•SANTIAGO* 

VERY  GOOD. 


IS  THE  FOLLOWING  TRUE  OR  FALSE? 
SANTIAGO  IS  IN  CENTRAL  CHILE. 

*TRUE* 

THAT'S  FINE. 


Because  the  student  is 
right,  SCHOLAR  goes  into  the 
data  base  entry  for  Santiago 
to  ask  more  questions.  (To 
go  into  an  entry,  the  entry 
must  have  the  main  topic  as 
a  Superpart,  i.e.,  it  must 
be  part  of  Chile.) 


THE  REGIONS  OF  CHILE  ARE  — . 

*1  DON'T  KNOW* 

THE  ANSWER  IS:  THE  CENTRAL  VALLEY,  THE 
ATACAMA  DESERT,  AND  THE  SOUTHERN  ANDES. 


SCHOLAR  h.s  run  out  of  the 
time  allotted  to  Capital 
and  so  goes  on  to  Regions. 
Time  is  allocated  on  the 
basis  of  the  number  of 
top-level  attributes  and 
their  I- tag  weights. 


THE  CENTRAL  VALLEY  IS  IN  CENTRAL  CHILE. 
IT  IS  ON  THE  COAST  OF  THE  PACIFIC. 

THE  ATACAMA  DESERT  IS  IN  CHILE  AND  PERU. 
IT  IS  BETWEEN  THE  PACIFIC  AND  THE  ANDES 
MOUNTAINS. 


Since  the  student  does  not 
know  any  Regions,  SCHOLAR 
gives  the  correct  answer, 
plus  some  information  about 
two  of  the  Regions, 


WHAT  IS  THE  LANGUAGE  OF  CHILE? 
♦SPANISH* 

YOU  ARE  CORRECT. 


Language  is  under  the  top- 
level  attribute  Population. 
SCHOLAR  goes  on  to 
Population  not  because  of 
time  but  because  the  student 
doesn't  know  any  Regions. 

That's  all  SCHOLAR  knows 
about  Language,  so  it  goes 
on  to  Reliqion,  which  is  at 
the  same  level  as  Language 
under  Population. 

The  rest  of  the  information 
about  Religion  is  above  the 
I-tag  level  for  the  first 
pass,  so  no  relate! 
information  is  printed. 


WHAT  IS  THE  PRINCIPAL  RELIGION  OF  CHILE? 
*1  DON'T  KNOW* 

THE  ANSWER  IS:  CATHOLICISM. 


Fig.  2b.  An  Annotated  Transcript  of  a  Dialogue  Between  Tutorial 
Mode  of  SCHOLAR  and  a  High-School  Student  in  Exp.  2. 
(The  Student's  Inputs  are  Between  Asterisks.) 
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(Boginning  of  Review 

THE  EASTERN  BORDERING  COUNTRY  OF  CHILE 
IS  — . 

*1  DON'T  KNOW* 

THE  ANSWER  IS:  ARGENTINA. 

THE  BOOT  DARY  OF  THE  EASTERN  BORDERING 
COUNTRIES  OF  CHILE  IS  THE  ANDES. 

THE  NORTHERN  BORDERING  COUNTRY  OF  CHILE 
IS  PERU. 

THE  NORTHEASTERN  BORDERING  COUNTRY  IS 
BOLIVIA. 

WHAT  ARE  THE  REGIONS  OF  CHILE? 

•CENTRAL  VALLEY  ATACAMA  DESSERT  AND  THE 
ANDES* 

"CENTRAL  VALLEY*  AND  "ATACAMA  DESERT" 
ARE  FINE. 

YOU  COULD  ALSO  MENTION  THE  SOUTHERN 
ANDES. 

I  DON'T  THINK  "DESSERT"  IS  CORRECT. 

THE  ATACAMA  DESERT  IS  IN  CHILE  AND  PERU. 
IT  IS  BETWEEN  THE  PACIFIC  AND  THE  ANDES 
MOUNTAINS . 

THE  SOUTHERN  ANDES  ARE  IN  SOUTHWESTERN 
SOUTH  AMERICA. 

THEY  ARE  IN  ARGENTINA  AND  CHILE. 

IS  IT  TRUE  THAT  THE  CENTRAL  VALLEY  IS 
IN  CENTRAL  CHILE? 

•YES* 

VERY  GOOD. 


Pass) 

On  the  second  peso,  SCHOLAR 
skips  to  the  first  piece  of 
information  the  student 
missed  earlier. 

Since  the  student  misses  it 
again,  the  same  related  in¬ 
formation  as  before  prints 
outs.  SCHOLAR  knows  that 
this  information  was  presented 
earlier,  and  we  may  revise 
SCHOLAR  so  that  it  asks  about 
the  related  information  here 
instead  of  presenting  it  again. 


The  student  missed  this  the 
first  time.  A  new  answer 
evaluation  module  points  out 
that  the  Andes  is  right  but 
more  specifically  it  is  the 
Southern  Andes. 

SCHOLAR  recognizes  Atacama  as 
the  Atacama  Desert  and  doe* 
not  understand  Dessert. 

Giving  related  information 
about  the  Atacama  Desert  was 
an  error  that  has  been  corrected. 


Because  the  student  has  named 
the  Central  Valley,  SCHOLAR 
goes  into  the  entry  for  Central 
Valley  to  ask  questions  about 
it.  Like  human  tutors,  SCHOLAR 
will  go  off  and  discuss  what  the 
student  answers  correctly.  The 
dialogue  continues  with  a  whole 
series  of  questions  about  the 
Central  Valley  and  the  Atacama 
Deuert . 


Fig.  2c.  An  Annotated  Transcript  of  the  Beginning  of  the  Review  Pass 
Between  Tutorial  Mode  of  SCHOLAR  and  a  High-School  Student 
in  Exp.  2 . 
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In  thi*  experiment  students  were  encouraged  to  ask  questions,  and 
one  of  these  appears  in  the  short  excerpt  shown.  This  excerpt  also 
illustrates  how  the  review  pass  skipped  over  questions  answered 
earlier,  and  went  into  the  material  in  greater  depth  than  the  first 
pass. 

Block-Test  Mode 


Instead  of  starting  with  questions  like  Tutorial  Mode,  Block- 
Test  Mode  presented  a  block  of  information  and  then  asked  questions 
about  the  facts  presented.  There  was  a  screen  over  the  teletype 
in  the  experiments  that  prevented  subjects  from  looking  back  at 
the  block  durinq  the  questions.  The  version  used  in  the  first 
experiment  asked  once  about  all  the  facts  presented  in  a  block. 

The  version  used  in  the  second  and  third  experiments  randomly 
selected  two  of  the  facts  presented  to  ask  about.  For  each  question 
the  student  missed,  this  latter  version  would  ask  about  an  additional 
fact  that  was  presented.  Then  it  would  ask  about  each  fact  the 
student  missed  once  more,  to  see  if  he  learned  the  right  answer. 

When  the  questioning  was  over  for  one  block,  Block-Test  Mode  pre¬ 
sented  the  next  block  and  then  asked  questions  about  it.  This  cycle 
continued  until  the  time  was  used  up  (except  in  Experiment  3  where 
time  was  divided  between  a  first  pass  and  a  review  pass,  as  in 
Tutorial  Mode). 
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Block-Teat  Mcde  selected  topics  in  the  same  order  of  importance 
as  Tutorial  Mode,  like  Tutorial  Mode,  it  went  down  deeper  into  a 
topic  until  time  wat  used  up  for  that  topic  or  there  was  no  important 
information  left  (a i  measured  by  I-tags) .  Unlike  Tutorial  Mode,  it 
did  not  pop-up  out  of  a  topic  when  the  student  made  a  mistake,  so 
that  Block-Test  Mole  went  into  most  topics  in  more  depth. 

Figure  3  shot  s  an  actual  protocol  about  Chile  between  Block- 
Test  Mode  and  a  h .gh- school  student  in  Experiment  3.  This  excerpt 
illustrates  all  tie  different  features  that  were  incorporated  into 
Block-Test  Mode  1 i  the  course  of  the  three  experiments.  In 
particular,  it  illustrates  the  presentation  of  a  block  of  facts, 
the  random  selection  of  facts  to  ask  questions  about,  the  inclusion 
cf  an  additional  jueation  for  each  question  the  student  missed, 
the  repeat  of  eaci  question  the  student  missed  within  a  block,  and 
the  review  pass  where  the  facts  the  student  answered  correctly  on 
the  first  pass  were  skipped  over.  Block-Test  Mode  in  Experiment  1 
did  not  have  most  of  these  features  as  will  be  explained  later, 
and  in  Experiment  2  did  not  include  a  review  pass. 
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Dialogue  Annotated  Comments 

CHILE  IS  A  COUNTRY.  Four  facts  from  the  data  base 

IT  IS  IN  SOUTHWESTERN  SOUTH  AMERICA.  are  presented  as  a  block  of 

THE  EASTERN  BORDERING  COUNTRY  IS  ARGENTINA,  information. 

THE  BOUNDARY  IS  THE  ANDES. 


WHAT  IS  THE  EASTERN  BORDERING  COUNTRY 
CHILE? 

♦ARGENTINA* 

VERY  GOOD. 


OF  SCHOLAR  then  asks  about  one  of 
the  four  facts  chosen  randomly. 

The  student  answers  correctly. 


WHAT  IS  THE  BOUNDARY  OF  THE  EASTERN 
BORDERING  COUNTRIES  OF  CHILE? 

* PACIFIC* 

WRONG. 


SCHOLAR  asks  about  a  second 
fact,  also  chosen  randomly. 

The  student  makes  a  mistake. 


THE  CORRECT  ANSWER  IS:  THE  ANDES. 


WHAT  IS  CHILE?  SCHOLAR  asks  qbout  more  than 

two  facts,  because  the  student 
missed  one  of  the  two  questions 
above . 

*A  COUNTRY  IN  SOUTH  AMERICA*  The  student  answers  correctly. 

THAT'S  FINE. 


THE  BOUNDARY  OF  THE  EASTERN  BORDERING 
COUNTRIES  OF  CHILE  IS - . 

♦ARGENTINA* 

THAT'S  NOT  RIGHT. 

THE  CORRECJT  ANSWF'  _S:  THE  ANDES. 


SCHOLAR  reasks  the  question 
the  student  missed  earlier. 

The  student  missed  again. 

SCHOLAR  only  reviews  each 
missed  question  once. 


Fig.  3a.  An  Annotated  Transcript  of  the  Beginning  of  a  Dialogue 
Between  Block-Test  Mode  of  SCHOLAR  and  a  High- School 
Student  in  Exp.  3.  (The  Student's  Inputs  are  Between  Asterisks.) 
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'’’HE  NORTHERN  BORDERING  COUNTRY  OF  CHILE 
IS  PERU. 

THE  NORTHEASTE1 N  BORDERING  COUNTRY  IS 
BOLIVIA. 

THE  WESTERN  COAST  IS  THE  PACIFIC. 

IT  IS  ON  THE  HUMBOLDT  CURRENT. 

THE  WESTERN  COAST  OF  CHILE  IS 

* PACIFIC* 

THAT'S  RIGHT. 

IS  THE  FOLLOWING  TRUE  OR  FALSE? 


DTHE  WESTERN  COAST  OF  CHILE  IS  NOT  ON  THE 
HUMBOLDT  CURRENT. 


♦FALSE : 
FALSE* 


"FALSE"  IS  FINE. 

I  DON'T  THINK  "FALSE"  IS  CORRECT. 


DTHE  NORTHERN  BORDERING  COUNTRY  OF 
CHILE  IS  — . 


*DONT  KNOEA* 
WRONG. 


THE  NORTHERN  BORDERING  COUNTRY  OF  CHILE 
IS  PERU. 


Bolt  Beranek  and  Newman  Inc. 


Having  reviewed  all  the  questions 
the  student  missed  the  first 
time,  SCHOLAR  presents  a  new 
block  of  information. 


SCHOLAR  is  confused  because 
the  student  typed  "False" 
twice.  It  treats  it  as  a 
wrong  answer,  and  goes  on. 
Nothing  disastrous  happens. 


SCHOLAR  doesn't  recognize 
"Don't  Know"  because  it  is 
misspelled,  and  treats  it  as 
an  error. 

The  dialogue  continued  through 
the  rest  of  the  first  pass. 


i 


Fig.  3b. 


An  Annotated  Transcript  of  the  Beginning  of  a  Dialogue 

Between  Block-Test  Mode  of  SCHOLAR  and  a  High-School 

Student  in  Exp.  3.  (The  Student's  Inputs  are  Between  Asterisks.) 
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{Beginning  of  Review  Pass) 

CHILE  IS  IN  SOUTHWESTERN  SOUTH  AMERICA. 

THE  DOUNDARY  OP  THE  EASTERN  BORDERING 
COUNTRIES  IS  THE  ANDES. 

THE  NORTHERN  BORDERING  COUNTRY  IS  PERU. 

THE  NORTHEASTERN  BORDERING  COUNTRY  IS 
BOLIVIA. 

IS  IT  TRUE  THAT  CHILE  IS  IN  SOUTHWESTERN 
SOUTH  AMERICA? 

*YES* 

YOU  ARE  CORRECT. 

WHAT  IS  THE  NORTHERN  BORDERING  COUNTRY 
OF  CHILE? 

♦PERU* 

THAT'S  RIGHT. 


THE  WESTERN  COAST  OF  CHILE  IS  ON  THE 
HUMBOLDT  CURRENT. 

THE  SOUTHERN  COAST  IS  THE  ANTARCTIC. 

THE  CAPITAL  IS  SANTIAGO. 

THE  REGIONS  ARE  THE  CENTRAL  VALLEY ,  THE 
ATACAMA  DESERT,  AND  THE  SOUTHERN  ANDES. 

THE  SOUTHERN  COAST  OF  CHILE  IS  — . 

*ANTARTICA* 

YOU  MISSPELLED  "ANTARCTIC". 

THAT'S  FINE. 

IS  IT  CORRECT  TO  SAY  THAT  THE  WESTERN 
COAST  OF  CHILE  IS  ON  THE  HUMBOLDT 
CURRENT? 

*YES* 

VERY  GOOD. 


Fig.  3c.  An  Annotated  Transcript  of  the  Beginning  of  the  Review 

Pass  Between  Block-Test  Mode  of  SCHOLAR  and  a  High-School 
Student  in  Exp.  3. 


SCHOLAR  presents  a  new  block 
because  the  student  answered 
both  questions  correctly. 

Again  it  skips  over  information 
the  student  answered  correctly 
on  the  first  pass. 


On  the  review  pass,  SCHOLAR 
skips  the  facts  the  student 
answered  correctly  on  the  first 
pass.  It  repeats  those  facts 
the  student  missed  or  was  not 
asked  about. 
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Testing  Procedure 

The  subjects  in  the  experiments  were  high-school  students  paid 
for  their  services.  They  participated  in  four  one-hour  sessions 
with  SCHOLAR  on  two  successive  Saturdays.  In  each  session  they 
learned  about  a  different  South  American  country.  They  took  a  pre¬ 
test  covering  all  four  countries  on  the  first  Saturday  before  any 
sessions.  Post- tests  on  the  two  countries  studied  each  Saturday 
were  given  the  following  Tuesday  or  Wednesday.  Thus  the  post-test 
scores  measured  fairly  lcng-term  retention. 

On  each  Saturday  the  student  learned  about  one  country  in 
Tutorial  Mode  and  one  in  Block-Test  Mode.  The  order  of  the  two 
modes  was  reversed  for  each  subject  on  the  two  days,  and  counter¬ 
balanced  across  subjects.  The  four  countries  were  Brazil,  Argentina, 
Chile,  and  Venezuela.  There  was  slightly  more  information  stored 
about  the  first  two  countries  than  about  the  last  two.  Thus,  Brazil 
was  paired  with  Argentina,  and  Chile  with  Venezuela,  so  that  if  a 
student  learned  about  one  in  Tutorial  Mode,  he  learned  about  the 
other  in  Block-Test  Mode.  This  counterbalancing  by  country,  day, 
and  order  determined  the  eight  different  orders  used  for  the  eight 
subjects  in  each  experiment. 

The  sessions  were  set  to  run  one  hour  in  overall  time  before 
SCHOLAR  ended  the  session.  The  amount  of  material  covered  in  an 
hour  varied  somewhat  depending  or  the  response  time  of  the  computer. 
These  variations  would  not  produce  any  systematic  differences 
between  the  modes,  because  the  order  was  the  counterbalanced 
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across  subjects.  However,  in  the  last  two  experiments,  there  may 
have  been  leas  material  covered  than  in  the  first  experiment, 
because  of  the  increased  load  on  the  computer. 

The  post-tests  consisted  of  50  questions  for  each  country  which 
fairly  exhaustively  covered  tho  information  the  student  might  have 
learned  in  the  session.  The  post-test  for  each  country  was  divided 
into  two  separately  administered  sections,  because  questions  in  the 
s  'cond  part  were  likely  to  give  tway  answers  to  questions  in  the 
first  part.  To  the  extent  possible,  questions  were  analogous  from 
country  to  country.  The  pre-test  consisted  of  20  questions  (a 
subset  of  the  50)  from  each  of  the  four  countries;  in  fact  the 
easiest  of  the  50  questions.  Based  on  the  pre-test  scores,  none 
of  the  students  knew  much  in  advance  about  any  of  the  countries. 

The  students  were  given  instructions  about  the  typewriter 
terminals  they  used  and  about  the  particular  mode  in  which  they 
were  about  to  run.  They  were  told  they  could  not  ask  questions  in 
Block- Test  Mode,  and  in  Tutorial  Mode  in  Experiment  3.  With 
Tutorial  Mode  in  Experiments  1  and  2  they  were  told  the  form  of 
some  types  of  questions  they  could  ask,  with  several  examples  of 
different  kinds. 

Questionnaires  were  given  along  with  the  pre-test  and  final 
post- test.  The  pre-test  questionnaire  was  used  to  make  sure  none 
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of  the  students  had  extensive  experience  with  computers  or  with 
any  particular  country.  There  were  also  several  questions  related 
to  whether  students  like  to  control  their  own  learning  or  not,  to 
see  if  these  answers  had  any  predictive  value  with  respect  to 
how  much  stt  '  *r;ts  would  learn  in  the  different  modes.  The  question¬ 
naire  given  with  the  final  post-test  interrogated  students  about 
the  effectiveness  of  different  aspects  of  the  two  modes. 


1 

l 


I 
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EXPERIMENT  1 

In  the  first  experiment  we  attempted  to  see  if  clearly 
different  forms  of  the  two  strategies  would  produce  a  significant 
difference  in  the  amount  of  knowledge  students  acquire.  Thus  the 
version  of  Block-Test  Mode  used  in  the  first  experiment  was  quite 
primitive.  It  proceeded  along  the  same  path  whether  the  student 
answered  correctly  or  incorrectly.  This  experiment  then  compared 
a  highly  adaptive  method  of  teaching  with  a  non-adaptive  method. 

Method 

In  Tutorial  Mode,  students  were  introduced  to  a  small  number 
of  question  types  they  could  ask:  the  types  were  "Where  is  X?" 
"What  is  X?"  and  "Tell  me  about  X."  where  X  could  be  anything 
like  the  Central  Valley  or  temperate.  This  type  of  question  is 
useful  when  SCHOLAR  uses  a  term  the  student  doesn't  know  about. 
Students  were  not  strongly  encouraged  to  ask  questions,  and  in 
fact  did  not  ask  many. 

The  version  of  Block-Test  Mode  used  in  Experiment  1  presented 
eight  pieces  of  information  rather  than  the  four  pieces  shown  in 
Fig.  3.  (Less  than  eight  were  presented  if  it  were  in  a  subtopic 
like  Central  Valley,  where  there  were  less  than  eight  available 
for  selection.)  It  then  went  on  to  ask  about  each  of  these  eight 
facts  in  the  order  presented.  It  corrected  the  student  when  he 
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made  a  mistake,  but  it  did  not  review  any  questions  the  student 
missed.  When  the  student  had  been  questioned  about  all  eight 
facts,  Block-Test  Mode  presented  the  next  block  of  eight  facts, 
and  so  on  until  the  hour  was  used  up. 


Results  and  Discussion 


The  average  difference  scores  between  pre-  and  post-tests 
for  each  subject  are  shown  in  Table  1,  broken  down  by  presentation 


mode.  To  analyze  the  results  of  the  experiment,  we  used  a  three- 
way  analysis  of  variance  based  on  the  difference  scores  with  rtiode 


was  only  one  observation  per  cell,  we  took  the  mean  square  of  the 


Of  the  main 


factors,  the  effect  of  mode  was  significant  (F (1,7)=17 . 53,  p<01) 


the  effect  of  subjects  was  significant  (F (7 ,7) =14 . 45 ,  p<01) ,  and  the 


interactions,  the  interaction  between  mode  and  order  was  signifi 


order,  F (7, 7) =2. 71.  The  significant  interaction  between  mode  and 


country  they  learned  about  on  each  day  better  them  the  first  country 


But  when  Block-Test  Mode  was  second,  this  effect  was  more  than 


offset  by  the  mode  effect,  and  so  the  difference  in  retention 


shows  up  as  an  interaction 
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The  two  effects  we  were  interested  in  were  the  effect  of 
mode,  where  Tutorial  Mode  was  clearly  superior  to  Block-Test  Mode, 
and  the  lack  of  any  interaction  between  mode  and  subjects.  Taken 
together  these  two  results  indicate  that  the  superiority  of 
Tutorial  Mode  was  common  to  all  the  students  and  not  just  to  those 
who  prefer  to  control  their  own  learning.  Hence  it  is  clear  that 
some  aspects  of  Tutorial  Mode  were  of  general  benefit  to  student's 
learning  of  factual  knowledge. 

In  the  first  experiment,  students  did  not  ask  many  questions 
in  Tutorial  Mode.  On  a  questionnaire  given  with  the  final  post-test, 
the  students  commented  favorably  about  Tutorial  Mode  and  particularly 
the  procedure  of  going  over  material  more  than  once.  In  contrast, 
they  said  Block-Test  Mode  gave  them  too  much  information  at  once. 

They  also  felt  it  was  very  helpful  to  get  information  related  to 
the  question  they  missed  in  Tutorial  Mode.  Based  on  these  comments, 
and  the  infrequency  of  questions  by  the  students,  the  superiority 
of  Tutorial  Mode  probably  was  due  to  its  reviewing,  to  the  excess 
of  information  presented  at  one  time  in  Block-Test  Mode,  and  to 
the  difference  in  the  way  the  two  modes  combine  questioning  and 
presentation. 
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EXPERIMENT  2 

In  the  second  experiment ,  we  decided  to  see  how  a  more  adap¬ 
tive  version  of  Block-Test  Mode  would  compare  with  Tutorial  Mode. 

In  view  of  the  student's  criticisms  of  Block-Test  Mode,  we  cut 
the  amount  of  information  in  a  block  to  four  facts,  and  included 
reviewing  of  questions  within  a  block.  At  the  same  time,  we 
strongly  encouraged  students  to  ask  questions  in  Tutorial  Mode  in 
order  to  see  if  this  would  leui  to  greater  learning  in  Tutorial 
Mode. 

Method 

Eight  new  students  from  the  same  high  school  were  used  as 
subjects.  Tutorial  Mode  was  improved  in  several  small  ways, 
but  functioned  essentially  as  in  the  first  experiment.  However, 
subjects  were  encouraged  to  ask  questions  and  told  about  a  wider 
variety  of  questions  they  could  ask,  with  many  more  examples 
given.  The  main  additional  types  explained  to  the  students  were 
"What?"  questions  like  "What  is  the  climate  of  the  Central  Valley?" 
and  true/false  questions  like  "Is  Lima  in  Chile?" 

Block-Test  Mode  was  changed  drastically  in  the  second  experiment. 
The  number  of  facts  presented  in  each  block  was  cut  to  four,  except 
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when  fewer  than  four  facts  were  stored  with  a  particular  subtopic. 
Instead  of  asking  about  all  the  facts  presented,  this  version  of 
Block-Test  Mode  randomly  selected  two  of  the  facts  presented  to 
ask  about.  It  asked  an  additional  question  for  each  one  the 
student  missed,  up  to  the  number  presented.  Then  it  would  review 
any  the  student  missed  or  did  not  answer  the  first  time.  After 
this  reviewing  within  the  block,  it  went  on  to  the  next  block. 
There  was  no  review  pass  through  the  material  as  there  was  in 
Tutorial  Mode,  nor  were  the  students  allowed  to  ask  questions. 

Results  and  Discussion 


The  results  for  Experiment  2  are  also  shown  in  Table  1, 
and  again  the  students  learned  much  more  in  Tutorial  Mode. 

Of  the  main  effects  in  the  analysis  of  variance,  the  effect 
of  mode  was  significant  (F (1, 7)=6 .18,p<.05) ,  the  effect  of 
subjects  was  significant  (F (7, 7 ) =5 . 16 ,p<.05) ,  and  the  effect 
of  order  was  not  significant  (F (1,7) *3 .4  8) .  Of  the  two-way 
interactions,  none  of  the  effects  were  significant:  for  subjects 
and  order  F(7,7)*.94,  and  for  subjects  and  order  F(7,7)»3.75» 
and  for  order  and  mode  F (1,7)  =  . 02.  There  was  more  variability  in 
the  data  for  this  experiment,  which  may  be  because  of  variability 
in  the  load  on  the  computer. 


BBN  Report  No.  2835 


Bolt  Beranek  and  Newman  Inc. 


In  this  experiment  we  eliminated  the  two  aspects  cf  Block- 
Test  Mode  that  the  students  complained  about;  i.e.,  the  excess 
of  information  in  a  block  and  the  failure  to  review  missed 
questions.  However,  Tutorial  Mode  still  showed  the  same  large 
advantage  over  Block-Test  Mode.  There  were  three  differences 
between  the  two  modes  that  might  contribute  to  this  difference 
in  effectiveness:  1)  the  way  they  combine  questioning  and  presen 
tation,  2)  the  allocation  of  time  between  a  first  pass  and  a 
review  pass  in  Tutorial  Mode  and  3)  the  option  for  students  to 
ask  questions  in  Tutorial  Mode.  Only  the  first  of  these  is 
inherently  different  between  the  modes. 

The  students  were  encouraged  to  ask  more  questions  with 
Tutorial  Mode  in  this  experiment,  and  in  fact  they  did.  Some 
asked  quite  a  few  questions  and  it  looked  as  if  question-asking 
was  a  detriment  for  some  and  a  benefit  for  others.  When  a 
student  went  off  asking  questions  on  his  own,  the  most  important 
information  often  was  not  covered.  But  when  the  student  used 
questions  for  clarification,  the  .ability  to  ask  questions  was 
probably  quite  helpful.  This  double-edged  nature  of  questions 
may  also  have  contributed  to  the  increase  in  variability  of  the 
data. 


We  had  expected  an  overall  increase  in  scores  between  the 
two  experiments  because  of  the  improvements  to  Block-Test  Mode 
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and  the  increase  in  student  questions  in  Tutorial  Mode.  The 
fact  that  there  was  no  improvement  in  Tutorial  Mode  may  have  beer 
due  to  the  double-edged  nature  of  questions.  In  Block-Test  Mode 
for  which  there  was  also  no  improvement,  it  may  have  been  due  to 
the  failure  to  ask  about  every  fact  presented.  Alternatively,  it 
could  have  been  due  to  tHe  particular  groups  of  subjects,  or  the 
fact  that  the  load  was  slightly  heavier  on  the  computer  in  the 
second  experiment,  which  would  lead  to  a  decrease  in  material 
covered.  Any  comparisons  across  experiments,  though  interesting. 


are  somewhat  tenuous . 


BBN  Report  No.  2885 


Bolt  Beranek  and  Newman  Inc. 


EXPERIMENT  3 

There  were  three  factors  then  that  could  have  contributec  to 
the  large  difference  between  the  two  modest  1)  the  way  the  modes 
combine  questioning  and  presentation,  2)  the  review  pass,  and 
3)  the  option  to  ask  questions.  In  order  to  pin  down  the  contri¬ 
bution  of  each  of  these  factors,  we  ran  a  third  experiment  con¬ 
verging  the  two  modes  toward  each  other.  The  first  factor  was 
inherently  different  between  the  modes  so  this  difference  was 
unchanged.  But  by  adding  a  review  pass  to  Block-Test  Mode,  and  by 
removing  the  option  to  ask  questions  in  Tutorial  Mode,  we  could 
parcel  out  the  effect  of  these  different  factors. 

Method 


Eight  new  subjects  were  used  from  the  same  high  school. 
Tutorial  Mode  was  the  same  as  in  the  previous  experiment,  except 
that  the  subjects  were  not  told  that  they  could  ask  questions. 

Block-Test  Mode  was  also  the  same  except  that  it  was  revised 
to  allocate  its  time  between  a  first  pass  and  a  review  pass  (see 
Figure  3c).  Thus  it  did  not  go  as  deeply  into  each  topic  as  the 
previous  version.  In  the  review  pass,  a  block  of  four  facts  was 
followed  by  a  sample  of  two  questions  just  as  in  the  first  pass. 
But  the  information  the  student  answered  correctly  on  the  first 
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pass  was  skipped  over  on  the  raview  pass.  A  fact  was  not  skipped 
over,  however,  if  it  was  presented  on  the  first  pass  but  not  asked 
about,  or  if  the  student  missed  the  question  about  it  the  first 
time  but  answered  correctly  on  the  review  question  within  the 
block.  The  review  pass  went  over  the  material  to  the  same  depth 
as  the  first  pass,  but  any  subsequent  passes  (if  time  was  left) 
went  somewhat  deeper. 

Results  and  Discussion 

The  results  for  Experiment  3  are  shown  in  Table  1  where  they 
can  be  compared  to  the  results  of  the  first  two  experiments.  Of 
the  main  effects  in  the  analysis  of  variance,  the  effect  of  sub¬ 
jects  was  significant  (F  (7 , 7) *7 . 67,  p  .01),  but  neither  the  effect 
of  mode  (F (1 , 7)  =  . 058 )  nor  the  effect  of  order  (F (1 , 7) =. 669 )  were 
significant.  None  of  the  interactions  were  significant:  for 
subjects  and  mode  F  (7 „ 7) =1.16 ,  for  subjects  and  order  F(7,7)=1.47, 
and  for  mode  and  order  F  (1,7)  =  .  058. 

It  is  clear  that  allocating  time  between  a  first  pass  and  a 
review  pass  eliminated  most  of  the  difference  between  the  modes, 
and  so  this  must  have  been  the  major  factor  contributing  to  the 
difference  between  the  modes.  The  residual  difference  between 
the  two  modes  is  a  measure  of  the  effect  of  the  way  the  two  modes 
combine  questioning  and  presentation.  There  seems  to  be  slight 
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benefit  to  the  tutorial  strategy  of  questioning  first  to  learn 
what  the  student  knows,  but  it  is  not  significant  with  such  a 
small  number  of  subjects. 

The  decrease  in  scores  for  Tutorial  Mode  from  Experiments 
1  and  2  to  Experiment  3  is  a  measure  of  the  effectiveness  of  the 
option  to  ask  questions.  Though  comparisons  across  experiments 
are  tenuous,  the  stability  of  the  scores  with  Tutorial  Mode  in 
Experiments  1  and  2  increases  confidence  in  the  comparison  with 
Experiment  3.  The  decrease  in  Experiment  3  is  a  small  effect, 
but  suggests  that  the  option  to  ask  questions  is  somewhat  bene¬ 
ficial. 
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DISCUSSION 

The  effectiveness  of  the  review  pass  for  the  student's  learning 
in  the  experiments  emphasizes  the  importance  of  Norman's  (1973) 
notion  of  web  teaching.  By  including  two  or  more  passes  in  Block- 
Test  Mode,  it  began  to  approximate  the  web-teaching  strategy  of 
building  a  basic  framework  on  one  pass,  to  which  new  material  is 
added  on  later  passes.  The  second  pass  repeats  what  the  student 
was  least  certain  about  on  the  first  pass,  so  that  the  repetition 
is  concentrated  on  the  most  difficult  things  to  remember.  It  is  a 
strategy  particular ly  suited  to  CAI,  because  computers  have  perfect 
memory  for  what  the  student  answered  earlier. 

In  the  experiments  there  was  little  benefit  from  using  the 
tutorial  strategy  of  finding  out  what  the  student  knows  first,  and 
then  adding  related  information.  This  may  reflect  the  fact  that 
all  the  students  knew  practically  nothing  about  South  American 
geography  to  start  with.  Therefore,  there  was  little  to  be  gained 
from  an  ability  to  skip  over  information  the  student  already  knew. 
The  real  effectiveness  of  the  tutorial  strategy  is  its  ability  to 
adapt  to  the  level  of  the  individual  student  (though  Tutorial  Mode 
only  partly  reflects  the  human  tutor's  ability),  and  the  students 
in  these  experiments  did  not  have  enough  knowledge  to  take  advantage 
of  this  capability. 
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The  failure  of  the  within  block  reviewing  using  Block-Test 
Mode  in  Experiment  2  to  benefit  student's  learning  probably  reflects 
the  spacing  effect  often  found  in  experimental  psychology  (Melton, 
1970).  By  spacing  repetitions  far  apart  in  time  as  in  Experiment  3, 
the  material  is  more  likely  to  be  doubly  encoded,  whereas  repeti¬ 
tions  spaced  close  together  as  in  Experiment  2  are  more  likely  to 
be  encoded  only  once.  Thus  the  material  widely  spaced  is  better 
remembered. 

The  method  used  here  can  be  extended  to  study  many  different 
aspects  of  teaching  methods.  It  can  be  used  to  investigate  teaching 
of  different  kinds  of  knowledge  other  than  the  factual  knowledge  in 
SCHOLAR'S  geography  data  base.  There  are  now  functioning  SCHOLAR 
systems  for  teaching  users  how  to  use  a  computer  text-editing 
system  (Grlgnetti,  et  al. ,  1974)  and  fcr  teaching  geography  with 
maps  (Collins  &  Warnock,  1974).  Another  system  is  planned  for  teaching 
functional  relations  in  geography  (Collins,  et  al,  1975)  .  For  example 

the  method  can  be  used  with  the  map  facility  by  comparing  how  well 
students  learn  the  same  material  with  and  without  maps.  The 
effectiveness  of  different  methods  may  vary  with  the  different  kinds 
of  knowledge  being  taught  or  with  different  students.  Beacuse  of 
its  inherent  flexibility,  the  teaching  method  SCHOLAR  uses  can  be 
geared  to  the  particular  kind  of  knowledge  or  even  left  to  the 
student's  choice.  This  is  one  of  the  great  potential  benefits  of 
generative  CAI. 
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Tho  fact  that  SCHOLAR  can  be  used  to  test  particular  aspects 
of  teaching  methods  makes  it  potentially  a  valuable  tool  for 
educational  research.  The  possibility  of  trying  out  •single 
modifications  in  teaching  strategy  to  see  their  effects  on 
students'  learning  rate  is  unique  to  SCHOLAR.  Human  teachers  of 
course  can  make  such  modifications  in  their  own  teaching  strategies, 
but  there  is  no  way  to  control  ell  the  other  factors  that  might  vary 
as  they  changed  strategy.  However,  any  specific  version  of 
SCHOLAR  is  a  fixed  system,  and  so  am  unbiased  comparison  can  be 
made  using  any  number  of  subjects.  In  this  way  the  accumulation  of 
systematic  knowledge  about  teaching  methods  can  begin  to  occur. 
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